The present invention relates generally to automated recognition of user input, such as handwriting recognition. There is a strong desire for automated systems that can recognize user input to improve the quality of human-machine (e.g. human-computer) interfacing. While the computer keyboard and mouse have become ubiquitous, keyboards and mice are inconvenient input devices in many applications. For example, mobile computing devices often are too small to provide a computer keyboard. As another example, some languages are not well suited to computer keyboards.
For example, some systems cannot accept Chinese characters, and users have been forced to use English or contrived systems such as Pinyin in using these systems. In Pinyin, Chinese characters are represented by phoneticized spellings in the Roman alphabet. Pinyin is cumbersome, however, because it requires the user to learn a new alphabet. Moreover, entering Pinyin is tedious if diacritical marks are included, since this requires additional keystrokes to define the diacritical marks. Diacritical marks are necessary to properly represent tonal information which conveys meaning in Chinese. Other systems allow input of Chinese characters by entering a combination of keyboard codes on a conventional QWERTY keyboard. Using these systems is also cumbersome, because it requires memorizing the sequence of keyboard codes for each character. Unlike keyboard entry, handwriting is quite natural. Accordingly, there exists a strong desire for handwriting recognition systems.
Various off-line types of handwriting recognition systems are known, sometimes referred to as optical character recognition. Off-line handwriting systems are limited in their use, however, since they cannot be used for real time applications, such as instant text messaging. Accordingly, off-line techniques fail to address needs such as mobile computing, instant messaging, and real-time computer input.
On-line handwriting recognition systems are known which can perform real-time or near real-time character recognition, but these systems are typically quite complex. For example, some known systems use hidden Markov models to represent each character within an alphabet to be recognized. Such systems typically require large amounts of memory and require significant computer processing power to perform character recognition. Hence, these systems can be slow in performing recognition or can have processing and/or memory requirements incompatible with power-limited portable computing applications.
Existing handwriting recognition systems have also typically been limited to recognizing characters from a single alphabet. Algorithms have sometimes been optimized for a particular alphabet, and thus cannot handle (or perform poorly) when mixed alphabets are used (e.g., combinations of English and Chinese characters).
Even greater challenges are presented by the problem of recognizing mathematical expressions. In mathematical expressions, not only are a large number of different symbols and alphabets used, but also the positional relationship of symbols is important as well. Most existing systems for entering mathematical expressions are tedious, requiring either the entry of special codes to define expressions (e.g. as in TeX) or requiring selection of each symbol from a series of menus (e.g. as in MathType).